Relaxed Clipping: A Global Training Method for Robust Regression and Classification

Robust regression and classification are often thought to require non-convex lossfunctions that prevent scalable, global training. However, such a view neglectsthe possibility of reformulated training methods that can yield practically solvablealternatives. A natural way to make a loss function more robust to outliers isto truncate loss values that exceed a maximum threshold. We demonstrate thata relaxation of this form of “loss clipping” can be made globally solvable andapplicable to any standard loss while guaranteeing robustness against outliers. Wepresent a generic procedure that can be applied to standard loss functions anddemonstrate improved robustness in regression and classification problems.